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DETAILED ACTION 



Specification 



1 . The disclosure is objected to because of the following informalities: 



• P. 3, line 11: "sys area" should have been "eye area" (per P. 10, line 10 of the 
specification); also for clarity consider changing "rate" to "aspect ratio" since 
clearly the intent is to determine the shape (elongated or not) of the eye area 
(per P. 3, line 12 of the specification) 

Appropriate correction is required. 

Claim Objections 

2. Claims 3, 5, 6, 15 and 16 are objected to because of the following informalities: 



• Claim 3, last line: for clarity consider changing "skin color region" to "skin color 
segmentation" (per P. 7, lines 3-15 of the specification, especially lines 3 and 
13-15) 

• Claim 5, line 4: for clarity consider changing "rate" to "aspect ratio;" do the 
same for claim 15 
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• Claim 5, line 4: per P. 10, line 10 of the specification "the sys area" should have 
been "the eye area"; the same applies to claim 15. [Note: for examination 
purpose "sys" will be interpreted as "eye"] 

• Claim 5, line 7: for clarity add "(MRB)" after "rectangle box;" do the same for 
claim 15 

• Claim 6, line 3: "slop" should have been "slope;" the same applies to claim 16 

• Claim 6, lines 5-6: for clarity consider changing "when eye areas of two eye 
candidate" to read "when the areas of the two eye candidates" (per P. 10, lines 
20-21); do the same for claim 16 

• Claim 6, line 10: "setting an luminance" should have been "setting a luminance;" 
the same applies to claiml 6 

Appropriate correction is required. 

Claim Rejections - 35 USC §112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 
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4. Claims 6-12 and 16-18 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

5. Claim 6, and similarly claim 16, recites the limitation "if the difference is with a 
predetermined range" in line 13 that is vague since it is not clear whether the difference 
is within or outside of the range; therefore the mete and bound of the claim cannot be 
ascertained. [Note: per P. 11, lines 12-13 of the specification, for examination purpose 
"with" will be interpreted as "within."] 

Claims 7-12 (depend from claim 6) and 17 and18 (depend from claim 16) inherit 
the same problems and are similarly rejected. 

6. Claim 7, and similarly claim 17, recites the limitation "face symmetric verification" 
in lines 2-3. This term is not a standard term in the art and is not defined in the 
specification; therefore the mete and bound of the claim cannot be ascertained. 

Claim 8 inherits the same problem and is similarly rejected. 

7. Claim 8 recites the limitation "symmetrical difference" in line 2. This term is not a 
standard term in the art and is not defined in the specification (the closest thing is 
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"symmetry value" in P. 1 1 , lines 20-21); therefore the mete and bound of the claim 
cannot be ascertained. [Note: per P. 1 1 , lines 20-21 of the specification, for 
examination purpose "symmetrical difference" will be interpreted as "symmetry value."] 

8. Claim 8 further recites the limitation "the selected one" in the last line. There is 
insufficient antecedent basis for this limitation in the claim. [Note: for examination 
purpose "the selected one" will be interpreted as "selected."] 

Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claims 1, 2, 13 and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Center, Jr. et al. (US 6,680,745), and further in view of Chai et al. 
("Face Segmentation Using Skin-Color Map in Videophone Applications," IEEE T. 
Circuits and Systems Technology, Vol. 9, No. 4, June 1999, pp. 551-564) and Luo (US 
6,151,403). 



11. Regarding claim 1, and similarly claims 13 and 14, Center discloses 
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• receiving an image data (in a YCbCr color space) • 

[Fig. 1: "camera", "locate face;" Fig. 5: "Current image." Note that 
the use of YCrCb space is taught by Chai, see below] 

• (using a Y component of the image data to) analyze out a motion region 
[Fig. 5: "motion detection;" Fig. 7; Col. 3, lines 25-36; Col. 5, lines 
25-60, especially 43-47. Note that a motion region is indicated by non- 
zero values. Note further that the Y component of Center, while 
representing the luminance, is from the YUV color space (Col. 3, lines 
50-51); the use of YCrCb space is taught by Chai, see below] 

• (using a CbCr component of the image to) analyze out a skin color region 
(Fig. 5: "Color Analysis;" Fig. 12; Col. 3, lines 25-36; Col. 7, lines 
5-7, 17-20 and 28-31. Note that a region with a non-zero value is a 
skin region. Note further that the color components of Center are from 
the YUV color space (Col. 3, lines 50-51); the use of YCrCb space' is 
taught by Chai, see below] 

• combining the motion region and the skin color region to produce a face 
candidate 

[Fig. 5: "Combination of Results;" Fig. 13; Col. 3, lines 25-36; Col. 7, 
line 50-Col. 8, line 21] 



Center does not expressly disclose using the YCrCb color space for skin color region 
detection, which is disclosed by Chai [P. 553, right column, last 8 lines]. The two 
references are combinable because they both have aspects that are from the same field 
of endeavor of face detection and the motivation for modifying Center by using YCrCb is 
provided by Chai in the same passage cited above. 



The combined invention of Center and Chai does not expressly disclose the following 
but Luo does 

• performing an eye detection process on the image to detect out eye 
candidates 

[Fig. 2a, refs. S6, S8, S14 & S16; Fig. 6 (multiple eye candidates; note 
that claim 14 requires at least two eye candidates); Col. 4, lines 35- 
61; Col. 5, lines 28-34; Col. 6, lines 53-55] 

• performing an eye-pair verification process, to find an eye-pair 
candidate from the eye candidates, wherein the eye-pair candidate is 
also within a region of the face candidate 

[Fig. 2a: refs. S20-S24; Figs. 6-8 & 10; Col. 6, line 53-Col. 8, line 3. 
Note that since all pairs of eyes are from the same flesh region (Fig. 
2a, ref. S20 and Col. 6, lines 64-Col. 7, line 1; note that a flesh 
region is considered a potential face candidate, per Col. 7, lines 27- 
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42), the eye-pair candidate is also within the same region (a face 
candidate) ] 

The combined invention of Center and Chai is combinable with Luo because they both 
have aspects that are from the same field of endeavor of face detection. 

At the time of the invention it would have been obvious to one of ordinary skill in the art 
to modify the combined invention of Center and Chai with the teaching of Luo in the 
manner recited above. The motivation would have been to further ascertain a region as 
a face candidate, as indicated by Luo in figure 10 and column 7, lines 27-42. [Note that 
while the main purpose of Luo is to eventually determine whether to eliminate a pair as 
an eye-pair candidate, this is achieved by determining whether a region containing the 
pair is a face (by verifying its symmetry). Therefore the determination that a pair is an 
eye-pair candidate also provides a strong piece of evidence that that the region 
containing the pair is a face candidate.] 

Therefore it would have been obvious to combine Chai and Luo with Center to obtain 
the invention as specified in claim 1. 

12. Regarding claim 2, the combined invention of Center, Chai and Luo further 
discloses 

• wherein a Cb value is between 77 and 127, and a Cr value is between 133 
and 173 

[Chai: P. 555, left column, lines 9-13] 
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***** 

1 3. Claim 3 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Center, 
Jr. et al. (US 6,680,745), Chai et al. ("Face Segmentation Using Skin-Color Map in 
Videophone Applications," IEEE T. Circuits and Systems Technology, Vol. 9, No. 4, 
June 1999, pp. 551-564) and Luo (US 6,151,403) as applied to claims 1, 2, 13 and 14 
above, and further in view of Hong et al. (US 6,999,634). 

14. Regarding claim 3, the combined invention of Center, Chai and Luo discloses all 
limitations of its parent, claim 1 . In addition, Center further discloses performing frame 
difference on the Y component for motion detection [Fig. 7; Col. 5, lines 25-30; note that 
a difference frame is also an image]. Further, by combining the motion detection result 
with the color detection result the probability that a region with a color similar to a face 
(but actually is not) that is detected using color alone is determined as a face candidate 
can be reduced if the probability of motion in the region is low [see Center: Fig. 13 and 
Col. 8, lines 6-22] and therefore can reduce false alarm (i.e., a region being erroneously 
determined as a face candidate — such a problem is well known in the art, see, for 
example, Col. 1, line 49-Col. 2, line 7 Cosatto et al. US 5,864,630) caused by using 
color alone; therefore by also using motion detection a drawback of the skin color region 
(segmentation) is compensated. 

The combined invention of Center, Chai and Luo does not expressly disclose the 
following but Hong does 
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wherein an infinite impulse response type (IIR-type) filter is applied 
to enhance the frame difference 

[Col. 7, lines 29-34. Note that since a difference frame is still an 
image, the IIR filter can be applied to reduce the noise; note further 
that the image is enhanced because the noise is reduced] 

The combined invention of Center, Chai and Luo is combinable with Hong because they 
both have aspects that are from the same field of endeavor of image processing. 

At the time of the invention it would have been obvious to one of ordinary skill in the art 
to modify the combined invention of Center, Chai and Luo with the teaching of Hong in 
the manner recited above. The motivation would have been to reduce noise while 
preserving edge/detail, as Hong indicates in Col. 7, lines 24-26. 

Therefore it would have been obvious to combine Hong with Center, Chai and Luo to 
obtain the invention as specified in claim 3. 

***** 

15. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Center, 
Jr. et al. (US 6,680,745), Chai et al. ("Face Segmentation Using Skin-Color Map in 
Videophone Applications," IEEE T. Circuits and Systems Technology, Vol. 9, No. 4, 
June 1999, pp. 551-564) and Luo (US 6,151,403) as applied to claims 1, 2, 13 and 14 
above, and further in view of Bloomberg (US 5,467,410). 
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16. Regarding claim 4, the combined invention of Center, Chai and Luo discloses all 
limitations of its parent, claim 1. In addition, Luo further discloses labeling flesh regions 
(i.e., face candidates) that have a size larger than a threshold [Col. 4, lines 12-25]. 

The combined invention of Center, Chai and Luo does not expressly disclose 
eliminating candidate regions with a relatively smaller label value. 

However, Bloomberg discloses labeling regions by their size and removing regions if 
their sizes (i.e., their labels) are smaller than a threshold [Fig. 3A; Col. 1 , lines 45-52 
and Col. 6, lines 6-14 (labeling by size and thresholding)]. 

The combined invention of Center, Chai and Luo is combinable with Bloomberg 
because they both have aspects that are from the same field of endeavor of 
segmentation. 

At the time of the invention it would have been obvious to one of ordinary skill in the art 
to modify the combined invention of Center, Chai and Luo with the teaching of 
Bloomberg in the manner recited above. The motivation would have been so that noise 
(such as regions that are too small to be a face candidate in face detection application) 
can be easily identified (e.g., by comparing the label to a threshold) and eliminated, as 
Bloomberg indicates in Col. 6, lines 16-17. 
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Therefore it would have been obvious to combine Bloomberg with Center; Chai and Luo 
to obtain the invention as specified in claim 4. 

***** 

17. Claims 5 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Center, Jr. et al. (US 6,680,745), Chai et al. ("Face Segmentation Using Skin-Color Map 
in Videophone Applications," IEEE T. Circuits and Systems Technology, Vol. 9, No. 4, 
June 1999, pp. 551-564) and Luo (US 6,151,403) as applied to claims 1, 2, 13 and 14 
above, and further in view of Abdel-Mottaleb et al. (US 6,263,1 13). 

18. Regarding claim 5, and similarly claim 15, Luo further discloses eliminating a 
connected region as a candidate of a flesh region (e.g., a face candidate) using the 
following criteria (the 2 nd and the 3 rd are with respect to a bounding ellipse, see Col. 4, 
lines 26-28): 

• if its area is out of a range [Col. 4, lines 12-14] 

• if it has a long shape [Col. 4, lines 28-35; note that the aspect ratio (i.e., rate) 
reflects the elongatedness of the object] 

• if it has proper density regulation (i.e., whether it has a small area but a large 
MRB) [Col. 4, line 31-35. Note that the compactness measure of Luo 
corresponds to the density regulation. Note further that Luo does not expressly 
disclose using a minimal rectangle box for the eye; but this is taught by Abdel- 
Mottaleb, see below] 
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Luo further discloses estimating the length and the orientation for an eye candidate [Col. 
4, lines 36-47]. 

The combined invention of Center, Chai and Luo expressly discloses neither eliminating 
regions as eye candidates using the criteria for face regions recited above nor enclosing 
an eye candidate with a minimal rectangle box (MRB) for evaluating density regulation 
(i.e., compactness). 

However, it would have been obvious to one of ordinary skill in the art to apply the same 
criteria for face candidates (as disclosed by Luo and described above), in addition to the 
length and the orientation, to determine whether regions are eye candidates. This is 
because eyes and faces both have their respective shape characteristics and by 
additionally incorporating similar criteria (for faces) to analyze the shape of regions to 
determine whether they are eye candidates, better accuracy can be expected since 
additional criteria have to be satisfied. 

In addition, Abdel-Mottaleb discloses analyzing shape (for face detection) by enclosing 
connected components with bounding boxes [Col. 4, lines 29-33] and two of the shape 
analysis criteria used there are aspect ratio (i.e., rate of candidate area) and 
compactness (i.e., density regulation) [Col. 4, lines 39-46; note that per the analysis 
above the criteria can be obviously applied to eye detection]. 
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The combined invention of Center, Chai and Luo is combinable with Abdel-Mottaleb 
because they both have aspects that are from the same field of endeavor of face 
detection. 

At the time of the invention it would have been obvious to modify the combined 
invention of Center, Chai and Luo with the teaching of Abdel-Mottaleb by using a 
bounding rectangle for potential eye candidates. The motivation for using a bounding 
box, rather than an ellipse as disclosed by Luo, would have been because fitting an 
ellipse is much more computation intensive; therefore computation cost can be reduced 
by using a bounding box, as one of ordinary skill in the art would have known. 

Therefore it would have been obvious to combine Abdel-Mottaleb with Center, Chai and 
Luo to obtain the invention as specified in claim 5. 

Allowable Subject Matter 

i 

19! Claims 6-12 and 16-18 would be allowable if rewritten to overcome the 
rejection(s) under 35 U.S.C. 112, 2nd paragraph, set forth in this Office action and to 
include all of the limitations of the base claim and any intervening claims. 



Application/Control Number: 10/671,271 
Art Unit: 2624 



Page 14 



20. The following is a statement of reasons for the indication of allowable subject 
matter: 

A. Regarding claim 6, and similarly claim 16, closest art of record does not disclose, 
teach or suggest all limitations. Specifically, 

• Lin et al. ("Locating the Eye in Human Face Images Using Fractal Dimensions," 
IEE Proc. Visual Image Signal Processing, Vol. 148, No. 6, Dec 2001, pp. 413- 
421) discloses verifying whether the orientation of an eye pair is within 45 
degrees [P. 416, Sect. 3.2, 1 st paragraph] 

• Held et al. (US 6,885,766) discloses verifying eye pair candidates using its 
orientation and the distance between the two eyes [Fig. 5 and Col. 8, lines 44-55 

• Luo (US 6,151,403) further discloses verifying that an eye pair candidate's 
orientation satisfying a different set of criteria [Fig. 7 and Col. 7, lines 1-9] and 
applying a bounding box to a face candidate for symmetry verification [Fig. 10 
and Col. 7, lines 27-42]. 

However, none of the above-cited references, alone or in combination, discloses, 
teaches or suggests comparing the difference of between the averaged luminance 
values of the center portion and the reminder of a luminance image in order to select an 
eye pair candidate, as recited in claim 6. 



Conclusion and Contact Information 
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21 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

• Cosatto (US 5,864,630) - discloses analyzing shape, color and motion for face 
detection [Fig. 1A] 

• Kim et al. (US 2004/0017930) - disclose combining the probability distributions 
(of pixels belonging to a face region) based on color and motion to locate 
candidate face regions [Fig. 1] 

• Lu et al. (US 2006/0233442) - discloses using color (in YCrCb color space) 
segmentation and shape analysis to detect face candidates and subsequently 
locating eyes from face candidates; motion is separately detected [Fig. 7; P. 6, 
paragraphs 116, 122-126] 

• Hwang et al. (US 5,832,101) - discloses applying MR to extract edges for 
subsequent motion detection [Fig. 3; Col. 7, lines 38-59] 

• Tao et al. ("Automatic Localization of Human Eyes in Complex Background," 
IEEE Int'l Symposium on Circuits and Systems, Vol. 5, 26-29 May 2002, pp. 
V669-V672) - discloses verifying eye candidates using criteria such as shape 
and area [P. V-670, left column, 1 st paragraph] 

22. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Yubin Hung whose telephone number is (571) 272- 
7451 . The examiner can normally be reached on 7:30 - 4:00. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew C. Bella can be reached on (571) 272-7778. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




